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Abstract — Recent results by Chen et al. and Polyanskiy et 
al. explore using feedback to approach capacity with short 
blocklengths. This paper explores Chernoff bounding techniques 
to extend the rate-compatible sphere-packing (RCSP) analysis 
proposed by Chen et al. to scenarios involving numerous retrans- 
missions and different step sizes in each incremental retransmis- 
sion. Williamson et al. employ exact RCSP computations for 
up to six transmissions. However, exact RCSP computation with 
more than six retransmissions becomes unwieldy because of joint 
error probabilities involving numerous chi-squared distributions. 
This paper explores Chernoff approaches for upper and lower 
bounds to provide support for computations involving more than 
six transmissions. 

We present two versions of upper and lower bounds for the 
two-transmission case. One of the versions is extended to the 
general case of m transmissions where m > 1. Computing the 
general bounds requires minimization of exponential functions 
with the auxiliary parameters, but is less complex and more stable 
than multiple rounds of numerical integration. These bounds also 
provide a good estimate of the expected throughput and expected 
latency, which are useful for optimization purposes. 



I. Introduction 



A. Previous Work 



It is well known that feedback can significantly improve the 
error exponent, [EQ-lEI. Under AWGN channel with noiseless 
feedback, the Shalkwijk and Kailath (SK) coding scheme 
achieves error probability with doubly exponential decay 0] 
121 . The SK scheme can be derived elegantly with the Elias 
result as shown in J6). The SK scheme is simple and efficient 
but requires full knowledge of the signal seen by the receiver 
to be communicated to the transmitter via feedback. 

On the other hand, Polyanskiy et al. Q show that full 
information through feedback is not necessary to achieve 
throughput close to capacity with low latency. Chen et al. J8) 
also show by simulation that a simple incremental redundancy 
scheme with feedback will allow a convolutional code with 
blocklength less than 200 to perform close to an LDPC code 
with blocklength close to 2000. 

The Rate-Compatible Sphere-Packing (RCSP) analysis was 
first proposed in J9) as an analytic tool to characterize the 
capacity-achieving potential of Hybrid ARQ systems. Both 
analysis and simulation results show that a simple feedback 
scheme using (ACK/NACK) with incremental redundancy 
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California Institute of Technology, under a contract with NASA. 



allows the system to achieve 90% of the capacity with an 
average latency less than 100 symbols. 

Achievability and converse bounds of variable length coding 
shown in Q reveal similar results of significant latency 
reduction when a noiseless feedback is presence. In particular, 
an example for the binary symmetric channel (BSC) shows 
that achieving 90% of the capacity requires only less than 
200 symbols. 

B. Main Contribution 

Chen et al. Q provide an approximation formula to com- 
pute the joint error probability of multiple transmissions with 
fixed step sizes. Computing the probability by numerical 
integration is also possible for small number of transmissions 
iflOl . As the number of transmissions grows, however, both 
the approximation and numerical method become unwieldy 
for optimization purposes. 

This paper provides lower and upper bounds on the relevant 
joint error probability for RCSP. The upper and lower bounds 
are given as infimums of closed form functions that require 
much less computation power. For the two-transmission case, 
two versions of upper and lower bounds are derived. The 
version where Chernoff bounds are used can be generalized 
to the m-transmission case. 

The results can be translated into upper and lower bounds on 
the expected throughput and expected latency. These bounds 
provide tight estimates of the expected latency and can be used 
to optimize the transmission rate and blocklengths for practical 
incremental redundancy schemes for general m transmissions. 
Examples in Section IIV-BI show that relaxing the bounds to 
several pairs of joint error events with a suboptimal but closed 
form auxiliary parameter already gives very sharp results. 

II. Problem Statement 

In this paper we consider a communication system under 
AWGN channel with a noiseless feedback. The use of the 
feedback link in our system is minimal: sending one bit of 
information for each block of forward transmission to confirm 
whether the message is received correctly. If the transmission 
is not successful, the transmitter will retransmit a block of non- 
repetitive incremental redundancy. The transmitter will attempt 
up to m transmissions (including the initial transmission). 



After the 7«th transmission the transmitter will restart from 
the initial transmission. 

The receiver uses a spherical bounded distance decoder 
defined as follows: consider an (M, n) code where M is the 
number of messages and n is the blocklength. The decoder 
maps the received sequence Y™ to the codeword that is within 
the decoding radius r (in terms of Euclidean distance). If there 
are more than one codewords or there is no codeword within 
the distance, the decoder will declare an error. 

For systems with feedback where only limited number 
of transmissions can be permitted, RCSP analysis (which 
assumes that the decoding radius r is what would be achieved 
by ideal sphere packing) provides practical guidance of the 
optimal transmission rate and blocklength. This was demon- 
strated in ifTOl . One of the issues in using RCSP to optimize 
transmission rates and blocklengths is the complexity of 
performing numerical integration to compute the joint error 
probability. In ifTol . exact computations were made, but this 
was only possible for up to six transmissions. In J9], an 
approximation formula based on the i.i.d. assumption gives an 
accurate estimate when the step sizes are large (which supports 
the i.i.d. assumption). 

This paper gives tight upper and lower bounds that allow 
a large number of transmissions and a relatively small step 
size. These bounds are in closed form (or are optimizations 
of closed form functions). The main results in the following 
sections are expressed in terms of the decoding radius 
of the codeword received at the ith retransmission. These 
results can then be evaluated by replacing with proper 
expression according to different assumptions. We always keep 
the cumulative distribution functions (CDFs) although they can 
also be bounded if desired. 

III. Main Results 

A. Bounds on Joint Error Event: Two-Transmission 

This section summarizes the upper and lower bounds of 
the joint error probability using a spherical bounded-distance 
decoder under AWGN channel. Some of the proofs are given 
in the Appendix. 

Let N\ = Ii be the blocklength of the initial transmission 
and Ii be the blocklength of the incremental redundancy trans- 
mitted at the ith transmission. The number of accumulated 
symbols at ith transmission is then Ni = N_x + Ii. Denote 
the bounded-distance decoding radius for the ith transmission 
as n. 

Suppose without loss of generality that the noise samples 
Zi are i.i.d. and Zj ~ A/"(0, 1). The error event Q of the ith 

transmission is given as Q = < ^ zf > rf >. The probability 

of each error event is simply the tail of a chi-square random 
variable: Pr(Ci) = Pr (x%. > rf). 

Because of the dependency between £j's, the probabilities of 
the joint events can only be expressed by integration. Take the 
two-transmission case for example, the joint error probability 
is given as 



/>oo 

Pr(Ci n Ca) = / Pr (x% > r\ - t) f xj (t)dt 

Jri 1 



(1) 



Pr 



(xl > r§ - t) f xji (t)dt + Pr (xl > r 2 2 ) (2) 



where f x 2 (t) is the density function of a chi-square distribu- 
tion with n degrees of freedom. 

We first summarize the two versions of upper and lower 
bounds for the two-transmission case. The first version of 
the upper and lower bounds uses the Chernoff bound. The 
following lemma states these upper and lower bounds for the 
two-transmission case. 

Lemma 1: 



Pr(Ci n C2 < inf } h , M 

^ S ' ~ 0<tt<l/2 (1 - 2u) N '/ 2 

Pr(Ci n Ca) > max(Pr(Ci) - toi,Pr(C 2 ) - w 2 ) 

e vrl p r ^ > ^ + 2v y2j 



where w\ = inf 

V>0 



W2 = inf 

0<v <l/2 



(l + 2«)^/ 2 
rl Pr (xl < (1 - Zv)rl) 



(3) 
(4) 

(5) 
(6) 



(1 - 2v) N */ 2 

Further bounding the expressions of Lemma \T\ is possible 
and can yield convex functions, but the u that optimizes these 
convex functions does not necessarily give the best bound 
in Lemma Q] Instead, we use a suboptimal but insightful 
parameter u* = (1 — N2/r%)/2. Let c 2 = r\j~N2- The upper 
bound, for example, then becomes 

cxp (-JV a (ca - 1 - In c 2 )/2) Pr ( X \ > rl/c 2 ) . (7) 

Assuming perfect sphere-packing (see Section II Vt , the radius- 
adjusting parameter c 2 is always greater than 1 ( hence u* < 
1/2 ) if the code rate is less than capacity. Note that our choice 
of u* does optimize the Chernoff upper bound for Pr(£2) and 
gives the expression exp (— A^ 2 (c2 — 1 — lnc2)/2). Equation 
© says that Pr(£i, C2) is approximately Pr(£2) multiplied by 
the probability of the first error event but with squared radius 
r\ divided by the factor c 2 . 

It is observed in |9| that the first few transmissions should 
have rates slightly above capacity to achieve the best expected 
throughput with feedback. The above Chernoff bounds give 
trivial results when the rate is above capacity, hence we 
provide a second version of the upper and lower bounds based 
on the results by Inglot ifTTI . We first state the theorem given 
by Inglot: 

Theorem 2 (Inlgot 477V): Let xl denote a random variable 
with central chi-square distribution and k degrees of freedom. 

For k > 2,r > k - 2, 



-M r ) (8) 



^(r)<Pr(x 2 fe >r)<-^— { _ 

where £fc(r) = exp {=p [r — k — (fc — 2) log(r/fc) + logfc]} 

Theorem |2] gives the following result for the two- 
transmission case. To simplify the equation let p = 
Pr(x? x > r|). 



Theorem 3: 
?H 7T~ 



\r\ - tY^-W^dt < Pr(Ci n C2) 



< p + inf_ A' / g(t)dt + / / 2 (t)dt (9) 

where J, 5, g(t) and K are described in detail in the Appendix. 

Although Theorem [3] cannot be generalized to the m- 
transmission case, numerical results show that the joint error 
probability on two events already gives surprisingly tight 
bounds (details are discussed in Section UV-Bt . Hence The- 
orem |3] may still be useful to obtain even tighter bounds 
especially when the rate is slightly higher than the capacity. 
The integral in Theorem [3] can be expressed in terms of an 
Appell hypergeometric function of two variables in closed 
form. See details in [fl~2). 

B. Bounds on the Joint Error Event: m Transmissions 

In the general case where m transmissions are allowed, there 
are m—1 step sizes 1%, . . . , I m and Ni = I\, JV» = i\T$_i + A 
for i > 1. The joint error probability can be expressed by (IT4V 

The following results give the upper and lower bounds 
based on Chernoff bounds for the m-transmission joint error 
probability: 

Theorem 4: Let Ui < 1/2, i = 1, 2, . . . , m be the param- 
eters for each use of Chernoff bound in the integral. Define 
hi,gi{u™) by the following recursion: 



h\ = ui, hi = hi— 1 + — 2/i 



i-lj 



5i 



•(l-2 Ul )-a 

3 -u i (l-2h i _i)r= 



9i = gi-ie 

Note the property that 1 - 2h t = Ylj<i(l — 2«j). We have 

P.^.y ^g^o^a (10 , 

Several versions of lower bounds can be obtained by dif- 
ferent expansions and the recursion formulas follow closely to 
those in Theorem |4] The following corollary gives an example 
of one specific expansion that yields a lower bound in a 
recursive fashion. The other formulas are omitted due to space 
limitations. See another example in Section HV-BI 

Corollary 5: Write Pr (ni<»< m Ci) = Pr(n 2 <i< TO Ci) 
Pr (r\2<i<mCi H Ci)- With the same recursion as in Theorem 
|U the lower bound is given in (TT3T >. 

IV. Application to RCSP 

For the RCSP analysis, we usually assume that at each 
decoding attempt, each received codeword will pack the sphere 
generated by the power constraint according to perfect sphere- 
packing. We also consider a more pessimistic assumption 
using Minkowski's lower bound. 

Consider an (M, n) code on the AWGN channel and let 
the SNR be r]. Assume without loss of generosity that each 
noise sample has a unit variance. Then the average power of a 



received codeword is less than P = n(l + 77). Sphere-packing 
seeks a codebook that has M codewords that represent the 
center of spheres (possibly overlapped) that are packed inside 
the n-dimensional ball with radius \Jn(l + rj). 

A. Pessimistic Sphere-Packing Under AWGN Channel 

This subsection uses the classic result by Minkowski in the 
sphere-packing argument: the packing density <f> > c2~™ for 
some constant c > 1 in K n . We use Minkowski's result to 
simplify our analysis even though the best known result scales 
as nc n 2~ n asymptotically lfl3l . 

The following theorem states that when r) > 1 the decoding 
time is finite a.s. and that the expected latency is also finite. 

Theorem 6: Assume that there exists a rate-compatible code 
with radii r, that at least achieves the packing density c2~™ 
in M™ and the r] > 1. Let Ni, a subsequence of N, be the 
blocklengths at each decoding attempt. Let the decoding time 
(also stopping time w.r.t. the natural filtration generated by 
{Zj}i) t = inf \Xn < r i} an d let L be the latency, then 
Pr(r is finite) = 1 and EL < 00. 

Since Minkowski's result is a lower bound on the packing 
density, Theorem [6] also holds under the perfect packing 
assumption in the next subsection. 

B. Optimistic Sphere-Packing Under AWGN Channel 

This subsection briefly reviews the argument of obtain- 
ing optimistic sphere-packing radii used in |9] and provides 
numerical examples based on the sphere-packing radii. The 
largest sphere -packing radius perfectly packs M spheres into 
the outer sphere. With this sphere-packing in mind, a conser- 
vation of volume argument yields the following inequality: 

w ,. T , , Vol (Outer sphere) 2 NJl + n) 
Vol (Inner sphere) < 1 '- rj < v 



M 



M 2 / N ' 



Based on the optimistic sphere-packing assumption, we 
give some examples of applying the bounds on joint error 
probability to obtain the latency versus throughput curve. 

Using the zero-error coding scheme described at the begin- 
ning of Section UU the expected latency EL and throughput 
ER t are given as follows (assuming Pr(£o) = 1) 

m 

EL = ER t = log M /EL. (11) 

i-Pr(n.j< m G) 

We apply Theorem|4]and its corollary to derive a tight lower 
bound on the joint error event. Rewrite the joint error event as 

f)i< m Ci = Cm \ C™ n f Di<m-i Ci) » which comes from the 
disjoint union of £ m : r\i< m (iU( m (ni< m _iCi) C = Cm- B Y °e 
Morgan's law 



Pr CiJ = Pr(Cm) - Pr (^C™ H |J Qj (12) 

m — 1 

>Pr(C m )- ^Pr(C„nCf) (13) 



\i=i 



-r £ 'j 1 1 V 3=1 / 



(14) 



Pr >max O.Pr f|C< 



inf 



gm-iKQPrfrgi < (l-2ft m -i)rj) 
(1 - 2/i m _i)'i/ 2 



(15) 



where the last inequality can be seen as the union bound on 
the second term of the first equality. Setting the parameter 
u = 1/2 - N m /(2rl l + 21og 2 M) in Theorem g] with m = 2 
gives a fairly tight lower bound despite being suboptimal, as 
shown in Fig [TJ We comment here that applying Theorem [3] 
may yield an even better bound but the evaluation of the bound 
is slightly more complex. 

To obtain an upper bound we rewrite the joint error proba- 
bility up to jth transmission (j < rri) as 

Pr ( f| CiJ = Pr Ci n Cm j + Pr ([) Q H C TO j (16) 

< Pr(c,„, d) + Pr (0 n O-i) - p r(0 n 0-i n Cm) (17) 

Applying Theorem |4] to the first two terms and Corollary 0to 
the last term gives an upper bound. 

We observed numerically that the inequality Pr(rii< m Ci) < 
Pr(Cm) gives surprisingly good bounds if the tail of a chi- 
square random variable is evaluated directly. Intuitively it 
says that given that the mth transmission is in error, most 
of the previous error events also occur with high probability. 
Although equation (JTvJ could give a better upper bound in 
some cases, the difference is negligible. 

Fig. [TJ shows the latency versus throughput curve for exact 
numerical integration, the upper bound (by lower bound on 
the error probability) of (TFJt and the lower bound (by upper 
bound on the error probability) of dlTt with a maximum 
of 5 transmissions and optimized step sizes based on iflOl . 
The channel SNR is 2 dB and the capacity is 0.6851. The 
number of information bits for each point from left to right 
are 16, 32, 64, 128, 256 respectively. The bounds are sharp as 
promised and the upper bound on the throughput curves up 
when the step sizes are too small such that the lower bounds 
on the error probability give trivial results. Using the inequality 
Pr(rij< m Ci) < Pr(Cm), Fig. [TJalso shows the lower bound on 
the throughput with the step size of 1 bit, which gives the best 
performance among all lower bounds. 

Fig.|2]shows the upper and lower bounds using (Q~3) and ([TTt 
with different step sizes (fixed for each number of information 
bits) at SNR 3 dB. The upper and lower bounds are already 
very tight when the step size is 10. The upper bound on 
throughput is above capacity and therefore not useful when 
the step sizes are too small. In a practical setting where the 
step sizes are optimized, however, the lower bounds may still 
provide useful insight. Also shown in the figure is the lower 
bound on the throughput with the finest increment (using the 
inequality Pr(rii< TO Cj) < P r (Cm))- The throughput of 1-bit 
increment follows similar trend as in Fig. [TJ 



Exact Calculation, Upper Bound and Lower Bounds. SNR = 2dB 
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Fig. 1 . The latency versus throughput curves of exact numerical integration, 
the upper and the lower bounds in the case of five transmissions on AWGN 
channel. Also shown in the figure is the best expected throughput by 
sending one bit increment at a time until the rate is well below capacity 
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log 2 M, Ii = l,m = 31og 2 M, i.e., the lowest rate is 1/3). The 



SNR is 2 dB and the corresponding capacity is 0.6851. 
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Fig. 2. The latency versus throughput curves of the upper and the lower 
bounds with difference step sizes at SNR 3 dB. When the step sizes are small 
the Chernoff lower bounds on the joint error probability give trivial results and 
hence the throughputs go beyond the capacity. As the number of messages M 
increases the upper and lower bound converges to the capacity. Also shown 
in the figure is the best expected throughput by using a step size of 1 bit. 



V. Conclusions 

This paper explores techniques to bound the performance 
suggested by the rate-compatible sphere-packing analysis. 
Using the Inglot Chernoff bounds, we derived two versions 
of upper and lower bounds of the relevant joint error event for 
the two-transmission case. The Inglot bounds are useful when 



the rate is slightly above capacity and the Chernoff bounds 
give cleaner expressions. We also presented general bounds on 
the m-transmissions case using Chernoff bounds. Numerical 
examples show that the bounds are tight when the step sizes 
are large enough. The achievable throughput with a step size 
of 1 closely approaches capacity with very low latencies. This 
well-known yet still exciting result brings the performance of 
the classic coding scheme proposed by Shalkwijk and Kailath 
to a more practical ARQ-like coding scheme, which only 
requires one bit of feedback at each transmission. 

We also show the finiteness of the decoding time and the 
expected latency using classic sphere-packing density result 
by Minkowski when t] > 1, which also implies finiteness with 
perfect packing assumption. 

Appendix 

This section provides the proofs of the Lemmas and Theo- 
rems in the previous sections. 

Proof of Lemma Q} Apply the Chernoff upper bound 

%x 



become very loose. We may, however, split the integral into 
two parts and bound them separately. 



2 Pr(x? 2 > r\ - t)f x%i (t)dt 

1 

(l-«)r! 

Pr(x? 2 > r\ - t)f x% (t)dt 



Pr(xl > r\ - t)f xir (t)dt 



(23) 
(24) 
(25) 



'(l-S)r 2 

<A"(ra,/ a ,JVi) / ( ) * gr a ,i a ,N 1 (t)dt+ f ' (t)dt. 

(26) 

Taking the infimum over 6 E (5, S) yields the upper bound: 

,(l-<5)rf 

2 f x%i (t)dt (27) 



inf_ K I g(t)dt - 

5e(8,5) J r j J(l-S)r. 



Pr(X > rj = Pr(e^ > e") J E[e^]e— to equation whgre R g{t) m funcdons rf 



O. Let z; 



i+lj ■ ■ 
then 



© < 



.,Zj),j > i and A^(r 



„2 _ r z T _ o 



■j5r ai J a ,iVi(t) 



^!/2-l( r 2 _ t) / 2 /2 

rl-h + 2-t ' 



e «(r|-ti) 

(1 - 2u)- /2 / 2 e- u(r ^* l) / y 2 (tx)*! 

i 



(18) K{r 2 J 2 ,Ni) 



e 2^2- 



2 W 1 /20 F/ a2-l)/2 r(Ari/2) - 



= (1 - 2u) 



-V2 e -ur£ 



E 



X4,5 1 ((l-2i l )r 



- E ^ 2 /2 



(l-27i)^/2(l_2ii) / i/ 2 e" 
e -url p r ^ > (1 _ 2u)rj 



(19) 



(20) 



(21) 



(22) 



Proof of Theorem [?] (sketch): Apply several rounds of 
similar steps (Chernoff upper bound and change of variable) 
as in the proof of Lemma Q] ■ 

Proof of Theorem® (sketch): Given the assumption, the 
decoding radii r, have the following inequality: 



r 2 >^i+Ac>l. 



(28) 



(1 - 2u) N */ 2 

where (|2TT > follows from a change of variable z[ = (1 + 
2u) x l 2 Zi. Taking the infimum over u < 1/2 gives the result. 

We sketch the proof for the lower bound due to space 
limitation. Observe that Pr(Ci n( 2 ) = Pr(Ci) - Pr«i H Q) = 
Pr(C 2 )-Pr(CinC 2 c ). Let Wl = Pr(CirX 2 c ), w 2 = Pr(CinC 2 c ) 
and finding the upper bounds of them yield the lower bound. 
The upper bound on w 2 follows from the above derivation by 
changing the integration interval from (ri,oo) to [0,ri]. For 
the upper bound on w\, apply the Chernoff bound with the 
form Pr(X <r) = Yy{er vX > e-' vr ) < E[e~ vX ]e vr . Taking 
the infimum over v > gives the result. 

■ 

Proof of Theorem \3] Applying the lower bound of 
Theorem [2] to equation (O gives the lower bound. 

For the upper bound, note that the denominator of the upper 
bound in Theorem |2] has a term r\ — I 2 + 2 — t. Hence the 
bound can only be integrate over r\ to r 2 — I 2 + 2. As the 
integration approaches r| — I 2 + 2 it's obvious that the bound 



Note that for rj > 1, Ni/rf = Ci > 1 for i large enough. 
Applying the Chernoff upper bound with the optimal param- 
eter u* gives a positive error exponent and the result follows 
from the Borel-Cantelli's lemma since e~ c ™, ne~ c " are both 
summable in n for some c' > 0. ■ 
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